@marklysze here is an example of failing completion call

Thanks @davorrunje, is it possible to get the AutoGen code that would have generated this?

As an update: I ran the params through OpenAI's API (response = completions.create(**params)) and it ran through okay and returned a function call.

With my agent's termination check it failed on checking the content for the termination keyword as content is None:
is_termination_msg=lambda x: True if "FINISH" in x["content"] else False

The exception: argument of type 'NoneType' is not iterable

Updating my termination expression corrected that:
is_termination_msg=lambda x: True if x["content"] and "FINISH" in x["content"] else False

@marklysze I marked the PR as a draft as I am not sure yet what kind of workaround is the best one. The issue seems to be on the OpenAPI side and it is more common with GPT-4o-mini than older models. Maybe we should remove names only if we get an exception? Even changing the system message a little bit helps sometimes. The list of possible workarounds is (https://community.openai.com/t/bizarre-issue-preventing-response-from-gpt-4o-mini-the-model-produces-invalid-content/875432):

• Remove tools and tool_choice args.
• Slightly reduce the length of the prompt, even by a couple of words.
• Add at least a second user message or an assistant message.
• Change the name or remove the name parameter (but that didn’t always work, depending on the prompt). Some names work, others don’t. ‘Gregory’ worked, for example.
• Change the message. Some work and some don’t. ‘Hello.’ didn’t work, but ‘Hello, friend.’ worked.
• Change to gpt-4o or gpt-3.5-turbo.

Thanks @davorrunje, appreciate the detail and the note on the draft.

It is definitely tricky because removing the name is a considerable change and could affect people's existing code - at the very least I'd suggest it is made as an option and defaults to existing behaviour of leaving it in.

From the list you provided, I think the third point would be the safest to do to minimise the impact on the validity of the messages. In the following linked LinkedIn post they added a message "DO NOT PRODUCE INVALID CONTENT" and that seemed to fix it! :).

LinkedIn Post on this being intermittent behaviour.

I would love to be able to replicate it, have you got any other examples that you can get to throw an exception?

I wonder if rather than change the messages to start with, we run inference and if it throws that specific exception then it adjusts the messages (perhaps with 3rd option above) and tries again up to x times.

Yes, I think we need to add a simple hack first and see how it goes. I see this error quite often when working with function calls and GPT-4o and GPT-4o-mini. I'll try adding a message as suggested first.

Okay, not sure if it's possible but if you do get the exception and are able to get the params["messages"] it would be interesting to see if it is replicated.

See how you go with the additional message.